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We are concerned with the end-to-end performance of image gathering, coding, and processing. The 
applications range from high-resolution television to vision-based robotics, wherever the resolution, effi- 
ciency and robustness of visual information acquisition and processing are critical. For our presentation 
at this workshop, it is convenient to divide research activities into the following two overlapping areas: 

1) The development of focal-plane processing techniques and technology to effectively combine 
image gathering with coding. The emphasis is on low-level vision processing akin to the retinal process- 
ing in human vision. Our approach includes the familiar Laplacian pyramid, the new intensity-dependent 
spatial summation, and parallel sensing/processing networks. Three-dimensional image gathenng is 
attained by combining laser ranging with sensor-array imaging. This work is summarized in the follow- 
ing five abstracts by T. Comsweet, G. Westrom, E. Kurrasch and R. Holben, R. Holben and 

G. Westrom, and D. Coon and A. Perera. 

2) The rigorous extension of information theory and optimal filtering to visual information 
acquisition and processing. The goal is to provide a comprehensive methodology for quantitatively 
assessing the end-to-end performance of image gathering, coding, and processing. Information t eory 
allows us to establish upper limits on the visual information which can be acquired within given con- 
straints, and optimal filtering allows us to establish upper limits on the performance that can be attained 
for specific tasks, even if these tasks require adaptive or interactive processing. This work is summarized 
in the remainder of this abstract. 

The performance of (digital) image-gathering systems is constrained by the spatial-frequency 
response of optical apertures, the sampling passband of photon-detection mechanisms, and the noise 
generated by photon detection and analog-to-digital conversion. Biophysical limitations have imposed 
similar constraints on natural vision. Visual information is inevitably lost in both image gathenng and 
low-level vision by aliasing, blurring, and noise. It is therefore no longer permissible to assume suffi- 
cient sampling as Shannon and Wiener could do in their classical works, respectively, on communication 
theory and optimal filtering for time-varying signals. Nevertheless, the digital processing algonthms ( or 
image restoration, edge enhancement, etc.) found in the currendy prevailing literature assume sufficient 
sampling, whereas image-gathering systems are ordinarily designed to permit considerable insufficient 
sampling. This fundamental difference between assumption and reality has caused unnecessary limita- 
tions in the performance of digital image gathering, coding, and processing. It has also led to unreliable 
conclusions about the correct design of image-gathering systems for visual information processing (as 
opposed to image reconstruction without processing, e.g., commercial television) and about the actual 
performance of image-coding schemes for tasks which involve digital image processing. 

Our analyses so far have shown that the combined process of image gathering and optimal process- 
ing can be treated as a communication channel if (and only if) the image-gathering degradations are 
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correctly accounted for. Correctly restored images gain significantly in fidelity (similarity to target), 
resolution (minimum discernible detail), sharpness (contrast between large areas), and clarity (absence 
of visible artifacts). These improvements in visual quality are obtained solely by the correct end-to-end 
optimization without increase in either data transmission or processing. Similar improvements can also 
be made in the resolution and accuracy of edge detection. Furthermore, if we implement the edge 
enhancement with focal-plane processing by properly combining optical response with lateral inhibition, 
it is possible to reduce data processing and transmission requirements and to improve robustness to 
noise. These results have encouraged us to extend our analyses to various image-coding schemes and the 
associated image restoration and feature-extraction algorithms. 
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